Papers with structural understanding
The Dog the Cat Chased Stumped the Model: Measuring When Language Models Abandon Structure for Shortcuts (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) can explain quantum mechanics and write sophisticated code, yet fail to parse sentences like "The cat that the mouse feared chased meowed" |
| Approach: | They propose a framework to distinguish structural understanding from semantic pattern matching . they use a set of 9,720 comprehension questions on center-embedded sentences . |
| Outcome: | a new framework shows that models lose performance when they abandon structural analysis for semantic associations. |
Optimizing Deeper Transformers on Small Datasets (2021.acl-long)
Copied to clipboard
Peng Xu, Dhruv Kumar, Wei Yang, Wenjie Zi, Keyi Tang, Chenyang Huang, Jackie Chi Kit Cheung, Simon J.D. Prince, Yanshuai Cao
| Challenge: | a common belief that training deep transformers from scratch requires large datasets is wrong . however, with proper initialization and optimization, the benefits of very deep transformer can carry over to challenging tasks with small datasets. |
| Approach: | They train 48 layers of transformers from pre-trained RoBERTa and 24 relation-aware layers from scratch. |
| Outcome: | The proposed scheme achieves state-of-the-art performance on a text-to-sql parsing benchmark . it uses 24 fine-tuned layers from pre-trained RoBERTa and 24 relation-aware layers from scratch . |
Exploring Graph Learning Tasks with Pure LLMs: A Comprehensive Benchmark and Investigation (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent studies focus on performance benchmarks without fully comparing LLMs to graph learning models. |
| Approach: | They evaluate off-the-shelf and instruction-tuned graph learning models across a variety of scenarios. |
| Outcome: | The proposed models outperform traditional graph learning models in few-shot settings, the authors show . their models out perform models with instruction tuning, and they show excellent generalization and robustness. |
MuSe: Multi-Stage Graph Reasoning via Vision-Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Graph Neural Networks (GNNs) and graph transformers are inadequate for tasks with limited generalization. |
| Approach: | They propose a multi-stage graph reasoning framework based on vision-language models that incrementally samples and visualizes task-relevant subgraphs. |
| Outcome: | The proposed framework outperforms existing benchmarks in Graph-related tasks. |
MolTRES: Improving Chemical Language Representation Learning for Molecular Property Prediction (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for chemical representation learning often lead to overfitting and limited scalability due to early convergence. |
| Approach: | They propose a framework to train Transformers on SMILES sequences to learn from structural examples and integrate external materials embedding to enrich molecular representations. |
| Outcome: | The proposed model outperforms state-of-the-art models on molecular property prediction tasks. |
HetGCoT: Heterogeneous Graph-Enhanced Chain-of-Thought LLM Reasoning for Academic Question Answering (2025.findings-emnlp)
Copied to clipboard
| Challenge: | graph neural networks capture structured graph information, but lack integration at the reasoning level. |
| Approach: | They propose a framework that leverages graph structural information to reason interpretable academic QA results. |
| Outcome: | The proposed framework outperforms sota baselines on OpenAlex and DBLP datasets. |
Jigsaw-Puzzles: From Seeing to Understanding to Reasoning in Vision-Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing vision-language models lack spatial reasoning capability, despite their ability to comprehend spatial arrangements and model structural relations. |
| Approach: | They propose a benchmark to evaluate vision-language models' spatial perception, structural understanding, and reasoning capabilities by minimizing reliance on domain-specific knowledge. |
| Outcome: | The proposed benchmark is based on 1,100 carefully curated real-world images with high spatial complexity. |
Datasets for Scientific Literature Understanding: A Survey (2026.findings-acl)
Copied to clipboard
| Challenge: | Empowering machines to understand scientific literature is crucial for accelerating scientific discovery and advancing the AI for Science paradigm. |
| Approach: | They propose a systematic taxonomy that organizes resources spanning structural understanding, text understanding, multimodal understanding and pre-training/instruction fine-tuning. |
| Outcome: | The proposed taxonomy organizes resources spanning structural understanding, text understanding, multimodal understanding and pre-training/instruction fine-tuning. |